Augmenting with Slot Filler Relevancy Signatures Data

Human readers can reliably identify many relevant texts merely by skimming the texts for domain-specific cues. These quick relevancy judgements require two steps: (1) recognizing an expression that is highly relevant to the given domain, e.g. "were killed" in the domain of terrorism, and (2) verifying that the context surrounding the expression is consistent with the relevancy guidelines for the domain, e.g. "5 soldiers were killed by guerrillas" is not consistent with the terrorism domain since victims of terrorist acts must be civilians 1. The Relevancy Signatures Algorithm attempts to simulate the first step in this process by deriving reliable relevancy cues from a corpus of training texts and using these cues to quickly identify new texts that are highly likely to be relevant. But since this algorithm makes no attempt to look beyond the relevancy cues, it will occasionally misclassify texts when the surrounding context contains additional information that makes the text irrelevant.